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Background / Context 

Early math achievement is a strong predietor of later sehool aehievement, and this predictive 
power is even greater than that of early literacy achievement (Claessens, Duncan, & Engel, 2009; 
Duncan et al., 2007). Research suggests that all ehildren are predisposed to use mathematical 
thinking (Balfanz, Ginsburg, & Greenes, 2003; Ginsburg, Choi, Eopez, Netley, & Chi, 1997), 
and reeent studies show that young children’s reasoning and knowledge in the areas of quantity, 
number, and spatial representation are malleable and sensitive to instruction (Claessens, Duncan, 
& Engel, 2009; Duncan et ah, 2007). However, many existing methods of teaching early math 
are falling short for too many ehildren — especially those from lower ineome households and 
English learners — and children who fall behind in math early on face long odds of catching up to 
their more mathematically proficient peers when it eomes to high-sehool graduation rates, 
college readiness, and ineome as adults (Duncan, et. ah, 2007; NAEYC & NCTM, 2010). 

The National Association for the Edueation of Young Children (NAEYC) and the National 
Council of Teachers of Mathematics (NCTM) have called attention to the need for challenging 
and effective early childhood math programs (2012; 2010). Yet most presehool teachers are not 
trained in early math eontent, the development of young ehildren’ s aequisition of math skills, or 
teaching strategies to promote math learning (Ginsburg, Eee, & Boyd, 2008), nor do they have 
access to high quality, eomprehensive eurricular content (Clements & Sarama, 2009). 

The eurrent study, a randomized controlled trial, explores how technology and educational 
transmedia resources ean enhanee prekindergarten math teaehing and learning in preschools, 
especially those serving children who may be at risk for academic difficulties due to economic 
and social disadvantages. This research is part of a multi-year summative evaluation of the CPB- 
PBS Ready To Eearn initiative, funded by the US Department of Education. A core aim of the 
initiative is delivering early math (and literaey) resourees on new and emerging digital platforms 
sueh as tablet computers, interactive whiteboards, and smartphones, as well as better-established 
technologies sueh as eomputers, video displays, and gaming consoles, and to ereate learning 
experiences that leverage the unique capabilities of these various teehnology platforms. 

Purpose / Objective / Research Question / Focus of Study 

The study’s goal was to understand how the integration of video, computer games and aetivities 
might impact children’s mathematics learning. The study also examined whether the experience 
enhanced teaehers’ beliefs about their own understanding of math and the potential of 
teehnology-enhaneed early math instruetion. The following researeh questions guided the study: 
(1) What is the impact of the PBS KIDS Transmedia Math Supplement and Technology & Media 
experience on young children ’s math learning and self-regulation? (2) What is the impact of the 
PBS KIDS Transmedia Math Supplement and Technology & Media experience on teachers ’ 
attitudes and beliefs about early math education, and using technology and media to support 
math learning? 

Setting: 

The study took plaee in early ehildhood edueation centers primarily serving children from low- 
income households in the New York City and San Erancisco Bay areas. To be eligible to 
partieipate in the study, instruction had to occur in English, and there had to be at least 10 four- 
to five-year-olds enrolled who were profieient in English. Teaehers also had to be willing to 
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participate in professional development in math instruction and technology integration, and be 
willing to use new elassroom teehnology. 

Population / Participants / Subjects: 

The study examined four- and five-year-old children from low-ineome households and their 
teaehers. The final sample eonsisted of 85 elassrooms (40 in New York City and 45 in San 
Franeiseo). Deseriptive information for the ehildren is available in Table 1. While all children in 
partieipating elassrooms were invited to engage in study activities, researehers randomly seleeted 
a sample of approximately 10 ehildren from eaeh classroom to participate in assessments. 

A sample of 157 teaehers in New York City and San Franeiseo partieipated in the study. The 
sample, which included lead teachers, teaeher assistants, and aides, comprised an ethnieally 
diverse group and more than half of teaehers (54%) reported earning a baehelor’s degree or 
higher. On average, teaehers reported approximately 13 years (SD=9.1) of teaehing experienee. 
There were no signifieant differenees between the teaehers in the three eonditions. 

Intervention / Program / Practice: 

The PBS KIDS Transmedia Math Supplement was eentered around PBS KIDS videos and digital 
games, played on a seleet set of learning teehnologies (i.e., presehool-speeifie interaetive 
whiteboards and laptop eomputers). The Supplement used a transmedia approaeh, foeusing on 
familiar charaeters, settings, and narratives aeross different media formats, sueh as digital video 
and interaetive online games. The supplement also ineluded non-digital materials, like books and 
foam shapes, and was designed to eomplement existing instruetional routines like eirele time and 
free play centers. 

The researeh team developed the Supplement by drawing on existing researeh of early ehildhood 
math instruetion and sequeneing (Clements & Sarama, 2009; Ginsburg, Greenes, & Balfanz, 
2003), the team’s understanding of typical early ehildhood math instruction from the 2011 
Context Study (EDC & SRI, 2011) and 2012 Prekindergarten Transmedia Math Pilot Study 
(EDC & SRI, 2012), and existing researeh on sueeessful teehnology integration in early 
ehildhood elassrooms (MeManis & Gunnewig, 2012). The target math skills ineluded: eounting, 
subitizing, reeognizing numerals; reeognizing, eomposing, and representing shapes; and patterns. 

Research Design: 

The study used a randomized eontrolled trial, three-eondition design: 

• PBS KIDS Transmedia Math Supplement: A third of classrooms (26) received the Math 
Supplement deseribed above as well as the interaetive whiteboards, laptop eomputers, 
broadband Internet aeeess, teehnieal support, and training and eoaebing required to enaet 
the Supplement. Teaehers also reeeived pre-study training and ongoing eoaehing support. 

• Teehnology & Media: A third of elassrooms (30) reeeived the same teehnology, teehnieal 
support, and aeeess to eoaehes as the Math Supplement teaehers, but did not reeeive the 
struetured eurriculum for integrating these resources into their instruction. 

• Business as Usual: A third of elassrooms (29) were instrueted to continue with their 
typieal instruetional routines and did not reeeive any teehnology or supports. They did 
not reeeive new teehnology to augment what might have been present before, and did not 
reeeive professional development or eoaehing during the study. 
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Data Collection and Analysis: 

Trained assessors conducted pre/post individual assessments using three assessments. The 
research team randomly assigned assessment order to avoid unintended order effects. For a fuller 
description of these measures, see Pasnik & Llorente (2013). 

The Research Based Early Mathematics Assessment (REMA short form) (Weiland et ah, 2012) 
served as a valid and reliable standardized assessment of children’s math skills. The 19 items in 
the REMA short version assess math skills considered essential in preschool and kindergarten 
(NGA/CCSSO, 2010; Clements & Sarama, 2009) — recognition of number and subitizing, and 
shape composition, and patterning (Weiland et ah, 2012). Each item includes a game-like 
activity that involves the assessor reading a verbal prompt and, at times, demonstrating with 
manipulatives. Children provide a verbal response, point, or engage with manipulatives. 

The Supplement-based Assessment (SBA) was developed by the research team to assess 
children’s understanding of the concepts in the PBS KIDS Transmedia Math Supplement. The 
SBA contains 20 items and targets children’s understanding of counting; number recognition and 
subitizing; shapes; and patterns. As in the REMA, SBA items involved game-like activities that 
required assessors to read a verbal prompt and children to provide a verbal response, point, or 
engage with manipulatives. Early childhood researchers with assessment experience developed 
the SBA and pilot-tested them on a sample of preschool children and revised as necessary. 

The Head-Toes-Knees-Shoulders (HTKS) measure (Ponitz et ah, 2008) is a validated measure of 
young children’s behavioral self-regulation. 

Findings / Results: 

As measured by the SBA assessment, children in the PBS KIDS Transmedia Math Supplement 
condition learned significantly more mathematics than children in both the Technology & Media 
and Business as Usual conditions. Children in the PBS KIDS Transmedia Math Supplement 
condition exhibited significantly higher SBA scores than children in the Technology & Media 
condition (1.43 points, effect size of 0.22, p<. 001). This means that being in the Math 
Supplement condition would lead to a 9% increase in percentile rank for an average student in 
the Technology & Media group. A marginally significant effect also was detected in REMA 
data. Children in the Math Supplement condition exhibited higher REMA scores than children in 
the Technology & Media condition (1.09 points, effect size of .15 and p < .06). Eurthermore, 
children in the PBS KIDS Transmedia Math Supplement condition exhibited significantly higher 
SBA scores than children in the Business As Usual condition (1.51 points, effect size of 0.24, p 
<.001). A marginally significant effect also was detected using the REMA, such that children in 
the PBS KIDS Transmedia Math Supplement condition exhibited higher REMA scores than 
children in the Business as Usual condition (1 .09 points, effect size of . 15 and p < .07). 

No significant effects were detected for Head-Toes- Knees-Shoulders scores. A summary table 
of the main effects model appears in the appendix. 

Preschool teachers who enacted the PBS KIDS Transmedia Math Supplement reported 
significant changes in their confidence and comfort with early math concepts and teaching with 
technology. See table 4 for the proportion of teachers who reported teaching math concepts with 
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technology. There was a significantly greater increase in the proportion of teachers who reported 
teaching shapes, numeracy, and patterns in the PBS KIDS Transmedia Math Supplement 
condition compared to teachers in the Technology & Media condition. For more details on the 
teacher outcomes, see Pasnik & Llorente (2013). 

Conclusions: 

These positive findings — children learning significantly more math and teachers expressing 
greater comfort with technology and confidence in their math teaching as a result of 
implementing the Math Supplement — support the notion that the thoughtful integration of well- 
designed digital resources coupled with adequate teacher supports can positively influence child 
and teacher outcomes. 

The sharp, cohesive curricular focus of the Supplement stands in contrast to a common approach 
of technology integration, which tends to leave teachers to select resources piecemeal and on 
their own. The study results provide further evidence of the weakness of such an approach. Not 
only did learning gains not appear in the Technology & Media condition, but teachers in this 
condition reported spending less time on math instruction than the Business As Usual condition, 
suggesting that the difficult work of integrating technology actually interrupted or interfered with 
typical instructional routines. 

Conversely, when teachers are prepared with the content knowledge and pedagogical experience 
needed to mediate children’s learning with technology, children are able to make use of the 
learning opportunities available through engagement with digital media. Digital transmedia’s 
potential to advance content-area learning for young children may be of greatest value for those 
children who are most in need of academic support. In other words, “The children who have the 
most to gain are the ones who gain the most.” Finally, the results demonstrate that recognizing 
teachers as the professionals they are is both a starting place and a commitment that must be 
sustained. 
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Appendix B, Tables and Figures 


Table 1. Total sample of children and descriptive statistics for age by condition. 


Condition 

Number of 
Children 

Mean Age 

SD 

Overall 

966 

4.55 

0.33 

PBS KIDS Transmedia Math 
Supplement 

307 

4.55 

0.34 

Technology & Media 

321 

4.55 

0.33 

Business as Usual 

338 

4.56 

0.33 


Table 2. Implementation Data Collections Activities 


Condition Components 

PBS 

Transmedia 

Math 

Supplement 

Technology & 
Media 

Business as 
Usual 

Interactive Whiteboard, 3 Chromebook 
laptops, Wireless Internet 




Printed 1 0-week PBS KIDS Transmedia Math 
Supplement Guide with sequenced activities 

u 



Printed 10-week guide with pointers to 
transmedia resources on PBS KIDS Lab and 
PBS KIDS websites 


Y 


Professional development for teachers on 
math and use of technology 

u 

Y 


Professional development for teachers on use 
of the PBS KIDS Transmedia Math 
Supplement 

u 



Professional development on math after study 
implementation 




Hands-on materials to enact 10-week PBS 
KIDS Transmedia Math Supplement guide 

u 



Website supporting guided transmedia 
experience with all videos and games in 
sequence 

u 



Ongoing technology support 

u 

Y 


Ongoing onsite coaching to support 
technology integration 

u 



Ongoing onsite coaching to support use of the 
PBS KIDS 

u 
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Table 3. Summary of PBS KIDS Transmedia Math Supplement impact estimates 


Impact Contrast 

Coefficient 

Std. 

Error 

Hedges’ 
g (Effect 
size) 

P 

Multiple 

Comparison 

Test* 

SBA 

(l)PBS KIDS Transmedia Math 
Supplement vs Business as Usual 

1.51 

0.302 

0.24 

<0.0 

01 

significant 

(2) Technology & Media vs 
Business as Usual 

0.08 

0.309 

0.01 

0.78 

9 

— 

(3) PBS KIDS Transmedia Math 
Supplement vs. Technology & 
Media 

1.43 

0.288 

0.22 

<0.0 

01 

significant 

REMA 

(l)PBS KIDS Transmedia Math 
Supplement vs Business as Usual 

1.09 

0.589 

0.15 

0.06 

4 

— 

(2) Technology & Media vs 
Business as Usual 

0.00 

0.587 

0.00 

0.99 

6 

— 

(3) PBS KIDS Transmedia Math 
Supplement vs. Technology & 
Media 

1.09 

0.571 

0.15 

0.05 

6 

— 

HTKS 

(l)PBS KIDS Transmedia Math 
Supplement vs Business as Usual 

-0.02 

1.432 

0.00 

0.99 

1 

— 

(2) Technology & Media vs 
Business as Usual 

-0.89 

1.460 

-0.05 

0.54 

2 

— 

3) PBS KIDS Transmedia Math 
Supplement vs. Technology & 
Media 

0.87 

1.370 

0.05 

0.52 

4 

— 
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Table 4, Proportion of teachers who reported teaching math concepts with technology 



Business as Usual 

Technology & 
Media 

PBS KIDS 
Transmedia Math 
Supplement 


Pre- 

Survey 

Post- 

Survey 

Pre- 

Survey 

Post- 

Survey 

Pre- 

Survey 

Post- 

Survey 

Counting 

59% 

56% 

33% 

84% ** 

51% 

90% ** 

Subitizing 

26% 

28% 

22% 

54% * 

16% 

64% ** 

Identifying 

numerals 

50% 

50% 

28% 

90% t 

47% 

84% ** 

Shapes 

50% 

44% 

24% 

66% ** 

40% 

84% ** 

Patterning 

46% 

46% 

33% 

80% ** 

49% 

84% * 

Addition 

30% 

34% 

22% 

78% ** 

33% 

76% ** 

Subtraction 

24% 

24% 

17% 

62% ** 

27% 

66% ** 

Measurement 

15% 

24% 

20% 

72% ** 

27% 

60% * 

Documenting/ 

analyzing 

results 

17% 

14% 

4% 

28%* 

47% 

34% * 


*p<.05, ** /><.001, tComparing PBS Transmedia Math Supplement to the Technology & Media 
Condition. 
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